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Introduction and summary 


In 20 1 1, the Center of American Progress released the first-ever attempt to 
evaluate the productivity of almost every major school district in the country. 

That project developed a set of relatively simple productivity metrics in order to 
measure the achievement that a school district produces relative to its spending, 
while controlling for factors outside a district’s control, such the cost of living and 
students living in poverty 

The findings of that first report were worrisome and underscored the fact that 
the nation suffers from a productivity crisis. The data suggested that low pro- 
ductivity might cost the nation’s school system billions of dollars a year. What’s 
more, too few states and districts tracked the hang that they received for their 
education buck. 

In this updated report, CAP uses these same metrics to once again examine the 
productivity of the nation’s school districts. We embarked on this second evalu- 
ation for a number of reasons. In many areas, education leaders continue to face 
difficult budget choices, and more than 300,000 education-related jobs have been 
lost since the start of the Great Recession.' At the same time, the advent of the 
new, more rigorous Common Core standards will demand that far more from 
educators, including better, tougher exams. In short, many educators are being 
asked to do more with less. 

But still, school productivity has not become part of the reform conversation, and 
with this project, our hope is to shine a light on how productivity differs across 
districts, as well as to identify key areas of reform. Moreover, for the first time, 
we conducted a special analysis of educational fiscal practices, diving deep into 
state budgeting approaches. We believe that if our education system had a more 
robust way of tracking expenditures, it could do more to increase productivity. 
Together with this report, we have also released analysis by CAP Senior Policy 
Analyst Robert Hanna on twin districts. Hanna’s analysis looks more closely at the 
programs and practices of more effective districts. 
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As noted in our previous report^ the emphasis on productivity does not mean 
that CAP endorses unfettered market-hased reforms. We continue to believe^ 
for instance, that school vouchers do not further the cause of public education. 
Nor do we argue that policymakers should spend less on education. Indeed, we 
believe neither of these approaches can solve the nations pressing education 
challenges, and together with this report, CAP is releasing a paper by Bruce 
Baker titled “America’s Most Financially Disadvantaged School Districts and 
How They Got that Way,” which looks at some of the severe inequities that 
plague our nation’s school system. 

The bottom line is that we believe policymakers and educators need to focus on 
what works in education and scale up those practices. This means focusing on 
effectiveness and on equity. We need, in other words, to look at both who gets 
education dollars and what they do with those dollars. 

What’s more, it is clear is that schools and districts can boost outcomes, and in 
recent years, a number of districts and states have significantly raised student 
achievement. But these success stories are not enough. We also need to figure 
ways to do more with what we have. 

Here is a summary of our most recent findings: 

* Low educational productivity remains a deeply pressing problem, with bil- 
lions of dollars lost in low-capacity districts. Thousands of school districts 
ranked poorly on at least one of our productivity metrics; hundreds showed low 
scores on all three of our productivity metrics. The lowest productivity school 
districts serve about 3 percent of the more than 41 million students covered by 
our study. (Note that the productivity rankings for 20 14 cannot be compared to 
the rankings in previous years, due to methodological limitations) 

* Some of the nation's most affluent school systems show a worrying lack of 
productivity. Our analysis showed that after accounting for factors outside of 
a district’s control, many high-spending districts posted middling productivity 
results. For example, only slightly more than one-third of the districts in the top 
third in spending were also in the top third in achievement. 
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* In some districts, spending priorities are clearly misplaced. Texas is one of 
the few states that report athletic spending at the district level, and the state s 
data suggest that more than 100 districts in Texas spend upward of $500 per stu- 
dent on athletics.^ A few districts in Texas spend more than $ 1,000 per student 
annually on athletics. To keep these numbers in perspective, the average unad- 
justed per-pupil operating expenditure in the state in 2013 was around $10,000. 

* State approaches to improving fiscal effectiveness vary widely. Only a few 
states, such as Rhode Island, currently take a weighted-student funding based 
approach to education, where money is distributed to schools based on stu- 
dent need. What s more, only two states, Florida and Texas, regularly rate the 
productivity of local school dollars. Some policymakers are taking on the issue 
of productivity, however, and some states, such as New York and Virginia, have 
taken smart capacity-building approaches. 

* States have failed to make fiscal equity a priority and large funding gaps 
exist across school districts. In our analysis, we calculated the expenditure dif- 
ference between a district that spends near the top and near the bottom in each 
state. This is a long-standing approach to measuring school finance inequity, and 
using the latest spending data provided by the federal government, we found 
that gaps among school districts remain high. In New Jersey, the difference 
between the wealthiest districts and the least wealthy district was $6,200, after 
adjusting for cost of living and student demographics. For this reason, we took 
significant steps in our report to control for funding disparities. 

* State budget practices are often inconsistent and opaque. Key expenditure- 
related definitions vary, and while almost every state now has a common chart of 
accounts — a type of budget dictionary — the specifics are not comparable across 
states. This means that what might count as curriculum spending in one state is 
most likely different than what counts as curriculum spending in another state. 

Plus, some state practices are difficult to follow. In Washington state, for instance, 
school districts are allowed to release two different sets of financial statements.^ 

The first set of statements is for the state s annual financial accounting system. The 
second set of statements meet a different set of accounting procedures. According 
to the state, the second set of financial statements are “considered to be special 
reports’ or ‘supplemental schedules’ and are not basic financial statements.” 
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Brief description of the productivity ratings used in this study 


This work builds on our 201 1 productivity study, and for the 
most part, we used the same methodology as in the previous 
report."* Specifically, the spending data come from the 201 0-1 1 
school year, the most recent year for which data are available. For 
achievement, we relied on the results of 2010-1 1 state reading 
and math assessments in elementary, middle, and high school. 
All three of our metrics use a green-to-red color-coding system, 
and the first two approaches use the matrix shown below to 


evaluate districts. The same color legend is used on the interac- 
tive comoanion website at www.americanDroaress.ora/ROI. 

ROI evaluation matrix 




Lowest 

achievement 

Medium 

achievement 

Highest 

achievement 

Lowest cost 

• 


• 

Medium cost 

• 

• 

• 

Highest cost 

• 

• 

• 


Basic Return on Investment index rating 
The Return on Investment, or ROI, index is a measure that rates 
school districts on how much academic achievement they real- 
ize for each dollar spent, relative to other districts in their state. 
To avoid penalizing districts where education costs are higher, 
we adjusted for a variety of factors, including cost-of-living 
differences and higher concentrations of low-income, non- 
English-speaking, and special education students. 

Adjusted Return on Investment index rating 
This measure uses the same approach as the Basic ROI but ap- 
plies a different statistical method, called a regression analysis, 
to account for the higher costs associated with serving larger 
concentrations of low-income, non-English-speaking, and 
special education students. The adjustments, or weights, used 
in the Basic ROI are not always sensitive enough to account for 
spending differences within states. 


Predicted Efficiency index rating 
The Predicted Efficiency rating measures whether a district's 
achievement is higher or lower than would be predicted after 
accounting for its per-pupil spending and concentrations of 
low-income, non-English-speaking, and special education 
students. Under this approach, a low-achieving district could 
get high marks if it performed better than predicted. Lower- 
ing academic expectations for students from disadvantaged 
backgrounds is not a policy position supported by CAP. 

As we have noted before, our measures are far from 
perfect, and individual district evaluations should be inter- 
preted with caution. The connection between spending and 
achievement is complex, and our methods cannot capture 
everything that goes into creating an efficient school system. 
Nor can we control for everything that is outside of a district's 
control, and our adjustments for factors such as poverty 
and students in special education are estimations and do 
not account for variations in severity and type within those 
demographic groups. 

Most of the variation in student achievement is within schools, 
and so district-level productivity results mask significant varia- 
tions in productivity within districts. Furthermore, one cannot 
compare productivity ratings across years due to the nature 
of our approach. Finally, we are aware that some of the data 
reported by states and districts have reliability issues, with 
agencies sometimes using inconsistent definitions and weak 
data collection practices. 

Despite these important caveats, we believe our district-level rat- 
ings use the best available methods and reveal important results. 
Our work has been guided by a panel of experts, who reviewed 
our approach and provided helpful feedback. Flowever, we take 
full responsibility for the methodology and resulting evaluations. 
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This report recommends the following: 


* States should build capacity for productivity gains through targeted grants, 
assistance teams, and performance metrics. When done well, performance 
metrics can provide local leaders with better information on their district’s pro- 
ductivity levels and also guide best practices. We also believe that states should 
consider creating grants that link increases in funding to improved student 
achievement and recommend that states build technical assistance teams that 
assist districts in increasing productivity. 

* Education leaders should improve accounting procedures and create a 
multistate initiative that will focus on building more robust education 
budgets. Educators can do a lot within their communities to make accounting 
and budgets more transparent and actionable. Some states have detailed school- 
level fiscal databases, which make it easier to evaluate local levels of equity and 
effectiveness. Other states such as Texas have made their fiscal database highly 
robust, which allow observers to easily compare district spending on discrete 
categories such as athletics. 

* Educators should also improve the quality of fiscal data across states, and 
the Common Core State Standards Initiative provides an example of how 
states can work together to create a stronger, more innovative education 
system. Something similar should be done within the fiscal space, with states 
coming together to develop more rigorous budgeting procedures. Such a group 
of state education leaders could create a common chart of accounts, set out best 
accounting practices, and generally build capacity. 

* States and districts should encourage smarter, fairer approaches to school 
funding, such as student-based funding policies. Policymakers should 
develop funding policies that direct money to students based on their needs. 
This will go a long way to give all schools and districts an equal opportunity 
to succeed. At the same time, the gross funding inequalities between school 
districts cannot be ignored, and policymakers must take steps to improve fiscal 
equity across schools, districts, and states. Specifically, we recommend weighted 
student funding, which has the potential to both solve equity and efficacy issues 
with current school funding approaches. 
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Background 


In late 20 13^ the federal government released the latest results from the National 
Assessment Education Progress, or NAEP. The exam is widely known as the 
Nations Report Card, and the test provides state-level student achievement scores 
in reading and math. The results from NAEP were promising, and a number 
of areas, including the District of Columbia and Tennessee, showed significant 
improvement in the scores ofboth theirfourth and eighth graders.^ Nationally, 
there was also a slight improvement in overall scores. 

What the results suggest is that education reform can make a difference. But 
there is an issue here, which is that we do not do enough to figure out what these 
achievement gains cost — and how we can get more for our education dollar. 

Think about it this way: While many areas have made real but modest gains on 
NAEP, per pupil spending in the U.S. has tripled in the past four decades, even 
after adjusting for inflation.'’ What’s more, the United States is not seeing the gains 
that other nations are making on international assessments. Over the past 14 
years, for instance, the United States has only shown a one-year gain on interna- 
tional tests.^ Compare that to other countries, such as Portugal, Liechtenstein, 
and Lithuania, which made gains at twice the rate of the United States and spend 
considerably less.* 

This report is not an argument that money does not matter in education. Lots of 
evidence suggests that dollars do make a large and significant difference, and some 
of the schools and districts that posted large achievement gains in recent years 
have done so by spending more money. Rather, we are arguing that money matters 
when it is spent wisely, and this project is an effort to gain more attention to the 
policy solutions around school productivity. 

Our effort does not ignore the fact that many school districts are often inequitably 
funded. Indeed, education funding in this country remains consistently unfair. 
Other reports by the Center for American Progress have shown, for instance, that 
before recent funding reforms, a 10 percent increase in the rate of low-income 
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students in California equaled a $411 drop in average teacher salary.^ (see the 
Methodology section on page 17 for a detailed description of how we adjust for 
poverty in our productivity analysis) 

But as a nation^ the United States cannot pretend that education dollars are end- 
less; and we must address the issue of spending effectiveness. This is not an aca- 
demic point, given lackluster education revenues. In fact, for many districts — both 
wealthy and poor — spending cuts are becoming a fact of life. More than 30 states 
have provided schools with less money than they did prior to the Great Recession, 
according to one recent analysis.^” Another survey showed that more than 80 
percent of school administrators now view their district as inadequately funded.” 
Consider just one state, Oklahoma: From 2008 to 2014, spending per student 
dropped a whopping 23 percent.” 

These funding rollbacks hurt kids, and in far too many cases, educators are elimi- 
nating effective programs and making short-sighted roll backs. Sequestration, for 
instance, has led to the loss of access to pre-K programs for nearly 60,000 chil- 
dren.” Others states and districts are scrapping or curtailing extended learning 
time programs, and last year, the Los Angeles Unified School District cut funding 
for its summer program by more than $1 million.” 

At the same time, educators are keeping in place programs that have shown little 
or limited research. For instance, many states and districts continue to pay extra 
for teachers to receive master’s degrees even though the practice has shown little 
efficacy. One recent study by researchers Raegan Miller and Marguerite Roza, in 
fact, found that the nation spends almost $15 billion a year on so-called “mas- 
ter’s degree bumps,” despite the fact that the programs do not boost student 
achievement.” 

'The bottom line is that much has changed since our last productivity report. An 
increasing number of states and districts have realized that productivity is a press- 
ing issue, and they have begun to focus on the issue. But more broadly, too much 
has stayed the same. Schools still spend money on less-than-effective programs, 
and as we will see in the following pages, educators still fail to track their educa- 
tion dollars to see how exactly they are being spent. But first, we will look at the 
state of education finance data. 
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Following the money 


There are many reasons for the uneven levels of productivity in our nations 
schools. In our previous productivity report, we looked closely at the issue of 
school management systems. In this report, we wanted to focus on a different 
aspect of productivity improvement — and that begins with data. More specifi- 
cally, it begins with tracking exactly how education dollars are spent. 

Educators have long bemoaned the lack of high-quality spending data. In the late 
1950s, a number of states tried to bring some clarity to the issue of education 
budgeting, and states such as Tennessee passed laws reforming their educational 
accounting procedures, mandating, among other things, a uniform, statewide 
chart of accounts. The idea behind many of these efforts to improve district was 
simple: Without robust fiscal reporting consistency and transparency, there can- 
not be robust fiscal management. Or as officials in Wyoming put it in the state’s 
School Budgeting, Accounting and Reporting Manual, the purpose of a common 
chart of accounts “is to ensure that school district fiscal data can be reported in a 
comprehensive and uniform manner.”^'’ 

Over the past few years, some states have pushed additional fiscal reforms. But 
they have not been enough. Our nation’s education funding system has grown 
far more complex over the years, and school systems today receive funds from a 
variety of sources, including from state, federal, and philanthropic sources. Plus, 
Americans’ expectations of school systems are much higher than ever before, not 
to mention that taxpayers want to know that their dollars are being spent wisely. 
Finally, recent technological developments allow for far more innovation and 
development within the budgeting space. 

As part of this report, we decided to look more closely at the issue of fiscal data, 
and as part of our study, we visited the education department websites of all 50 
states and the District of Columbia and searched for charts of accounts and other 
fiscal information. We also did a special analysis on intrastate chart of accounts 
and looked closely at state fiscal databases. What was clear was that in some areas. 
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states and districts cannot easily compare their spending. Definitions are loose; 
standards differ. Perhaps worst of all^ many states and districts simply have no idea 
what kind of hang they are getting for their education huck. 

The issue is compounded hy the way in which states and the federal governments 
hand out their education dollars. Most districts receive education funds through 
so-called categorical programs, which direct money to specific programs or initia- 
tives. As a consequence, the dollars come to the district with their own set of 
policies and procedures. What’s more, some districts will set up a separate budget 
for these programs in order to adhere to the regulations, hut when a district has 
a separate budget for a program, it makes it difficult to examine overall spending. 
Furthermore, these categorical programs create incentives for local leaders to cre- 
ate varying budgets and accounting procedures, which, as we will see below, often 
make it hard to closely track spending and connect that spending to results. 


A not-so common chart of accounts 

As part of this study, we looked closely at the charts of ac- 
counts of California, Michigan, and Alaska.'^ We found that 
these states' definitions for a variety of spending proce- 
dures and functions differ, making it difficult to do a com- 
parison across states using only a chart of account code. 

Consider, for instance, the definition of principal duties. 
California nests principal duties under the school adminis- 
tration classification. However, Michigan goes a step further 
to create a code specifically for the "office of the principal" 
within the school administration category. There are other 


issues too. Alaska has a stand-alone expenditures category 
for library books. Michigan does not. California provides a 
clear definition of "workstations," but neither California nor 
Alaska do. Michigan provides a line item for "advertisement" 
expenditures. The other two states do not.'® 

These sorts of differing definitions make it difficult to track 
spending across state lines, and if someone wanted to eas- 
ily compare spending on library books in Greenville, Alaska, 
to Greenville, California, they would have to dig deep into 
the data in order to create meaningful comparisons. 
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Recent efforts to address 
budgeting practices 


Over the past few years^ some education leaders have realized that more needs to 
he done to improve education budgeting practices. Some outside organizations 
have heen developing their own approaches, and a few years ago, the Council of 
Great City Schools published a report on Managing for Results in America's Great 
City Schools, which includes self-reported data from 61 member districts on more 
than 300 data points relating to big-city school operations.^® 

States have also been looking at the issue of fiscal budgeting more closely. In 2012, 
for instance, Connecticut Gov. Dannel Malloy (D) announced the passage of a set 
of education reform initiatives that included a mandated chart of accounts.^ The 
budget project is currently in development and should be released by the 2014-15 
school year. 

A task force in Kansas has also recently made recommendations on fiscal prac- 
tices. The effort started when a school efficiency task force in the state tried to 
detail administrative cost-savings strategies.^^ But the task force found that it was 
difficult to differentiate what qualifies as instructional spending as opposed to 
administrative costs, and so the group recommended forming another group, 
which would create a common definition of instructional spending. 

Rhode Island has established one of the more promising reforms. To address 
a chronically inequitable and ineffective funding system, the state reformed 
its school funding system in 2010, and as part of that effort, the state required 
districts to enact a uniform chart of accounts. But the state did more than build 
a robust fiscal system, as education researcher Ken Wong has noted.^^ The state 
also connected the fiscal data to other programmatic activities. Wong writes that, 
under the new statewide fiscal system, “district and state policymakers will be able 
to compare, for example, how a particular district or school is spending their state 
and local dollars to support math or reading learning.” This means, in other words, 
that education leaders will be now be able to examine specific instructional prac- 
tices, such as technology- enhanced math classes, in terms of the dollars spent. 
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Another crucial part of Rhode Island’s fiscal reform is that it relies on a form of 
weighted student funding?^ This approach to funding has a number of names — 
including “student-based budgeting” — and it ensures that funding “follows every 
student to whatever public school he or she attends.”^'^ There are a number of 
benefits to the approach. It improves transparency and local autonomy by reduc- 
ing the reliance on categorical programs. Just as important^ the approach makes 
school funding more equitable. As Wong writes of the Rhode Island fiscal strategy, 
“the student success factor in the funding formula creates incentives for schools 
and districts that receive children who come from at-risk backgrounds.”^^ 

When it comes to productivity, the Rhode Island reforms offer an important 
lesson that robust fiscal data alone are not enough. Expenditure data also need 
to be connected to information on practice, programs, and student outcomes. In 
this way, leaders should be able to find answers to questions such as: What is the 
ROI on the district’s new math curriculum? What is the productivity increase of 
extending the school day by one hour? Are iPads in science classes paying off? 
And perhaps most important: Can leaders take action? Another way of under- 
standing the idea of nuanced expenditure data is to imagine it like this: If educa- 
tors have access just to fiscal data, they can understand only how their districts 
spends its money. But if educators have linked fiscal, programmatic, and outcome 
data, they can understand how their district can spend their money in order to 
gain better results. 
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Recent state productivity efforts 


As part of our 2014 ROI reporp we also decided to look more closely at state 
approaches to improving productivity and as part of the study we visited the 
education department wehsites of all 50 states and the District of Columbia and 
searched for any efforts or initiatives to improve fiscal effectiveness. Additionally 
we conducted interviews with some key leaders in the fields hoth in and outside of 
state government. 

The results of that research varied widely. The most common approach was for 
states to create a taskforce or write a one-off report. For instance^ a half-dozen 
states hired school finance experts Picus Odden & Associates to write an analysis 
and offer solutions to their productivity issues. The firm relies on an evidence- 
hased approach, which identifies school-level reforms that are research-hased 
and then creates a spending model for implementing these reforms in practice. 
Wisconsin, Wyoming, Virginia, North Dakota, West Virginia, and New Jersey con- 
tracted with Picus Odden & Associates to create such reports. 

There are issues with the Picus Odden & Associates approach, however. First, one- 
off efforts do not generally provide long-term solutions to the issues that haunt 
school productivity. In many cases, the problems are systemic and need deep and 
forward-thinking solutions. Moreover, some economists, such as Eric Hanushek, 
argue that the approach of Picus Odden & Associates is limited because it looks 
at potential gains of certain programs rather than taking a holistic view. The Picus 
Odden & Associates approach, argues Hanushek, is “essentially a selective review 
of the published literature on program effects.”^'’ Finally, the approach is limited 
because it’s executed at the state level and does not address local needs or goals. 

Other states have taken different approaches. Since 2003, Virginia has been 
conducting district-level fiscal analysis in order to “realize cost savings in non- 
instructional areas in order to redirect those funds towards classroom activities.”^^ 
Outside consultant reviews — think of them as a type of efficiency SWAT team — 
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conduct the analysis for the districts, and both the state and the district share 
the costs. Since the program began, more than 30 districts in Virginia have gone 
through the program with more than $40 million in estimated savings.^* 

Part of the success of the Virginia program lies within the fact that it is tai- 
lored to the context of specific districts. In 2012, for instance, Arlington Public 
Schools went through the efficiency review process, and the report recom- 
mended, among other things, that the district drop its “full 30-minute, duty-free 
lunch period to teachers.” As the document noted, “Food services leadership 
indicated that there is a requirement to provide a full 30-minute, duty-free lunch 
period to teachers, which influences their lunch period scheduling; however, 
based on research conducted by the review team, this does not appear to be a 
federal, state or local requirement.”^’ 

New York State has taken a more targeted approach, and the state recently began 
offering District Management Efficiency grants. Gov. Andrew M. Cuomo (D) 
began the program — which, according to the grant language, “rewards school dis- 
tricts that have implemented innovative strategies to improve the overall efficiency 
of school district management, while maintaining or improving student achieve- 
ment.”^” Districts in the state can apply for the grant by writing up a plan for cost 
savings in their districts. In 2013, four districts received the grants after identifying 
more than $20 million in cost savings.^' 

“The simple truth is that New York State spends more money per pupil than any 
other state in the nation, yet continually lags behind in student performance,” 
Cuomo said when the state announced the grant winners.^^ “We cannot continue 
to ask taxpayers for more and more money; rather our education system must 
become more efficient and focus spending on student achievement.” 

Many of the New York grant award-winners have created innovative programs. For 
instance, the state gave the Middletown School District more than $100,000 to 
create a self-check-in and check-out library system.^^ The next-generation library 
system relies on radio frequency identification to track books, and it is estimated 
to save the district more than $500,000 over two years.^'* 
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As for actually evaluating the productivity of schools and districts, today only 
two states, Florida and Texas, regularly examine education hang for the huck. The 
Texas program — known as FAST — began in 2009, and it uses a sophisticated 
modeling technique to control for factors outside of a district’s control such as 
student poverty. The program evaluates schools and districts each year and offers 
a set of “smart practices” gathered from local interviews. Some of the recommen- 
dations include basic leveraging of technology, such as moving to Internet-based 
phones. Others are more sophisticated — for example, some districts have used 
outside vendors to build instructional capacity. Florida has been running a pro- 
ductivity evaluation program for years. But this year, the state decided to reform 
the approach to better align with the state’s data system. The new approach is cur- 
rently awaiting approval from the State Board of Education. 

Perhaps the one thing that is clear is that all 50 states and the District of Columbia 
currently have the technical ability to provide school and district leaders with 
additional tools to improve productivity. For one, all the states have data on stu- 
dent outcomes, and all are required under the No Child Left Behind Act of 2001, 
or NCLB, to evaluate student performance. At the same time, all the states have 
data on expenditures. What states fail to do, however, is to combine these data 
in any meaningful way and provide district leaders with the tools that they need 
to make better choices. Some states have been exploring the idea, and in 2010, 
Oregon ran productivity evaluations on all of its districts and provided them to 
the districts on an informal basis.^^ 

In Colorado, the administration of Gov. John Hickenlooper (D) has floated a 
proposal to make school funding more transparent and require all schools and 
districts to place their budgets online.^* Hickenlooper hopes to bring the proposal 
to the state legislature this year and recently told a reporter: 

So far, no state’s ever had total transparency on how their tax dollars are spent to 
every school ...So you can create a Web site where every dollar to every school 
every day is tracked. How much goes to the teacher, how much goes to their 
pension, how much goes to the bureaucracy, how much goes to maintaining the 
facility. Really focus on making each school an enterprise.^^ 
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Methodology: Our approach to 
measuring educational ROI 


One of our goals in this project was to measure academic achievement relative 
to a district’s educational spending, while controlling for factors outside their 
control, such as cost of living and degree of student poverty. This workhuilds 
upon a great deal of previous workhy ourselves and others. For a history of our 
approach, including our technical advisory hoard, please consult our 2011 report, 
“Education ROI.” 

In this year’s analysis, we produced productivity evaluations for more than 
7,000 districts that enroll more than 80 percent of all U.S. students.^* We were 
unahle to produce results for Alaska, the District of Columbia, Hawaii, Kansas, 
Maine, Montana, Nebraska, New Hampshire, North Dakota, Oklahoma, South 
Dakota, and Vermont. Hawaii and Washington, D.C., are single-district jurisdic- 
tions, so within-state comparisons were not possible. Alaska, Arizona, Kansas, 
Maine, Montana, New Hampshire, North Dakota, Oklahoma, South Dakota, and 
Vermont did not have enough comparable districts. 

Spending data came from the Local Education Agency Finance Survey, or F-33, 
produced by the federal government’s National Center for Education Statistics, 
or NCES. These data are from the 2010-11 school year, the most recent year for 
which complete data are available. Since that time, many districts may have taken 
steps that might have significantly changed their efficiency ratings. 

As we noted in our previous report, we used the “current expenditures” cat- 
egory, which includes salaries, services, and supplies.^’ It does not include capital 
expenses, which tend to have dramatic increases from year to year and thus are 
unreliable for comparisons. The expenditure data include money from all revenue 
sources, federal, state, and local. 
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We restricted our study to districts with at least 300 students who took tests 
and that offered schooling from kindergarten through the 12th grade. We also 
excluded districts classified as a charter school agency, state-operated institu- 
tion, regional education services agency, supervisory union, or federal agency. 
Furthermore, we relied on a federal database, which placed further restrictions 
on the size of the district. To protect student privacy, the U.S. Department of 
Education reported exact student proficiency data for districts with more than 
300 student test takers.'*^ Data from New York City Public Schools were also 
aggregated into a single district. And to ensure that we had a sufficient number of 
comparable districts in each state, we included states only if more than 50 percent 
of their districts were covered by our analysis. If more than 50 percent of students 
in a district were special education students, that district was also removed the 
analysis. 

We also relied on NCES to calculate district-level demographic indicators for the 
2010-11 school year, the number of students receiving free and reduced-price 
lunches, the number designated as English language learners, and the number that 
participate in special education. 

Many districts did not report demographic data for the 2010-11 school year, 
necessitating the use of proxies. If a school district was missing a demographic 
indicator, we substituted data from the 2011-12, 2009-10, or 2008-09 school year. 
In no instance did we use proxies for achievement or expenditure data. 

Achievement data came from the U.S. Department of Education, which collects 
data from the states on district-level student outcomes. We used these data to 
create an achievement index, developing a score for each district by calculating 
the percent of students who scored proficient or above on the state assessment in 
reading or math in third grade through eighth grade and high school for the 2010- 
1 1 school year.'*^ 


Our three productivity measures 

As noted in our previous report, to emphasize the complexity of measuring a 
district’s productivity, we offer three different approaches to measuring productiv- 
ity rather than a single ranking. The companion website to this report allows the 
public to compare districts in a state using each of our metrics and also provides 
comparisons of school systems with similar demographics and size. The site also 
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details each district’s achievement and spending data. We used shades of colors 
when ranking the districts to emphasize the fact that we did not evaluate districts 
against an external benchmark hut rather on their relative performance. 


Basic Return on Investment index rating 

This measure rates school districts on how much academic achievement they get 
for each dollar spent^ relative to other districts in their state. 

Because it costs more to educate certain populations than their peers, we adjusted 
the expenditure data for students in special programs, such as students who 
receive subsidized lunches or are in special education. This is a common practice 
in school finance research, and we derived the weights by calculating the average 
weight used in a half-dozen research studies and policy papers.'^'* Based on those 
calculations, we used a weight of 1.4 for free and reduced-price lunch, 1.4 for 
English-language learners, and 2. 1 for special education. 

To understand how this works, consider the following. Research indicates that 
each student who qualifies for a subsidized lunch costs about 40 percent more 
to educate. So, for each additional student in the free and reduced-priced lunch 
program, we subtracted 40 percent from the district’s per-student spending. 

To adjust for cost-of-living differences, we used the Comparable Wage Index, or 
CWl, a measure of regional variations in the salaries of college graduates who 
are not educators. Lori Taylor at Texas A&M University and William Fowler at 
George Mason University developed the CWl to help researchers fine-tune edu- 
cation finance data to make better comparisons across geographic areas.’^^ We used 
adjustments from 2011, the most recent year for which data are available. 

To calculate the adjusted costs for each district, we created a needs index designed 
to measure how much additional funding a school district should have received 
based on its students in special programs, including the percentage of students 
in the subsidized school lunch program, special education students, and English- 
language learners, or ELL. We created the index by multiplying the number of 
students in these special programs by their respective weight. We then divided 
the weight by the enrollment to get the average additional amount of funding that 
a given school district should have received. To avoid penalizing districts with 
greater needs, we then divided the raw per-pupil expenditure by the weighted 
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index to produce the amount of money a district would have spent if it had no stu- 
dents in special programs. Finally, we adjusted this measure by the CWI to make it 
comparable across different geographic localities. 

We then distributed districts in each state into three equal tiers based on their 
position on the achievement index, with the highest achievers in the top tier and 
the lowest achievers in the bottom tier. We also divided the districts into three 
equal tiers based on their adjusted expenditures, with the highest adjusted spend- 
ers in the top tier and the lowest adjusted spenders in the bottom tier. We then 
used an evaluation matrix to assign colors to each district based on their achieve- 
ment tier relative to their spending tier, with green being the most productive and 
red being the least productive. 

The matrix rewards districts that had low spending and high achievement relative 
to other districts in their state. So if a district was in the top third of achievement 
and the bottom third in spending, it would receive a rating of green. 


ROI evaluation matrix 




Lowest 

Medium 

Highest 


achievement 

achievement 

achievement 

Lowest cost 

• 

• 

• 

Medium cost 

• 

• 

• 

Highest cost 

• 

• 

• 


Adjusted Return on Investment index rating 

This measure uses the same approach as the Basic ROI rating but applies a differ- 
ent statistical method, called a regression analysis, to account for factors outside a 
district’s control, such as the added costs of educating low-income, non-English- 
speaking, and special education students. The adjustments, or weights, used in 
the Basic ROI are not always sensitive enough to account for spending differences 
within states. For example, states might provide districts with additional funding 
for students in special education, and thus a weight of 2. 1 for a student in special 
education might be too high. 
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In this approach, we predicted what a district would spend relative to other 
districts in the state. We ran the regression models separately for each state to 
account for variation within each state s educational financing system. Here is the 
process depicted as an equation: 

ln(CWI adjusted ppe)= free lunch + % ELL+ p^ % Special Ed + e 

We predicted each district s spending based on the percentage of students in spe- 
cial programs, including the percentage of students receiving free or reduced-price 
lunches, the percentage designated as ELLs, and the percentage who participate in 
special education. Thus, we predicted how much more or less the school district is 
spending than what we predicted it should he spending — also known as a resid- 
ual — and we used this as our measure of spending. 

We then divided the districts into three tiers based on how much more or less the 
district spent than what we predicted it should have spent. Districts with lower- 
than-predicted scores went into the lowest tiers, and those with higher-than-pre- 
dicted scores into the highest tier. 

We then used the achievement index to separate the districts into three tiers, as in 
the Basic ROI rating. Finally, we assigned each district a color on the evaluation 
matrix based on its placement on the achievement and predicted-spending tiers. 


Predicted Efficiency index rating 

This measure is significantly different than the first two measures. 

The first two measures rate districts based on the achievement school systems 
produce compared to their expenditures after controlling for factors outside the 
district’s control. In contrast, the predicted efficiency measure does not compare 
achievement to spending. Instead, the approach rates districts on the results of 
their predicted achievement after controlling for factors outside their control. This 
distinction is important. The first two approaches attempt to measure how much 
“bang for the buck” a school district gets. This third approach attempts to elimi- 
nate the effects of spending and other factors such as students with additional 
needs and then evaluates districts by how much more or less achievement the 
district produced than would be expected. 
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Technically, then, this approach does not evaluate districts against an evaluation 
matrix nor does it weight or predict the amount that a school district spends on 
education. Instead, we used a regression analysis to predict what achievement a 
district should have relative to other districts in the state given its spending and 
percentage of students in special programs. 

To calculate this estimate, we used a production function, a type of regression 
analysis that examines the relationship of inputs to an output, and we predicted 
the achievement index as a function of the district s cost of living-adjusted 
per-pupil expenditure, the percentage of students participating in the free and 
reduced-priced lunch program, the percentage of students who are ELLs, and the 
percentage of special education students. 

This approach is shown in equation form helow: 

achievement = -i- ln(CWI adjusted ppe) + free lunch + P3 % ELL + % 

Special Ed -1- e 

To control for differences in state finance systems, we calculated individualized 
production functions for each state. Then, after predicting each district’s achieve- 
ment, we divided the results into six hands and awarded colors to districts that 
produced higher or lower levels of achievement than would he expected, with 
green being the most productive and red being the least productive. Districts with 
negative scores — or those that produced a lower level of achievement than would 
be expected — were given the least desirable rankings. 

One of the limitations of the Predicted Efficiency index is that districts with high 
overall achievement can receive low productivity scores. That is not the case with 
the first two productivity approaches. The measure also adjusts academic expec- 
tations for students from disadvantaged backgrounds. While this is an accepted 
research practice in the education policy community, the Center for American 
Progress opposes the lowering of academic expectations as a matter of policy. The 
reasons are both philosophical and practical. They are philosophical because we 
do not believe that a country that declares that everyone is created equally should 
have lower educational standards for students who are from low- income families 
or speak English as a second language. And they are practical because we believe 
that unless schools have high-academic expectations, it will not be ensured that all 
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students — regardless of family background — will succeed. But as we researched 
various productivity measures^ we found that this approach provided important 
insight into a district’s productivity and helped provide a more well-rounded 
understanding of its overall efficiency 


Important caveats 

As we have note before, one of the aims of this project is to bring attention to 
the large variance in productivity within states, and while we believe that our 
district-level evaluations rely on the best available methods — and show impor- 
tant results — we caution against making firm conclusions about the ratings of an 
individual district.’^ We also encourage readers to closely examine the data and 
our approach to evaluating productivity. 

The literature on productivity is limited, and there is a lot we do not know about 
the relationship between spending and achievement. It appears, for instance, that 
the link between outcomes and money is not always linear. In other words, even in 
an efficient school system, the first few dollars spent on a program or school might 
not have the same effect as subsequent expenditures, with additional dollars not 
boosting outcomes as much as initial investments. We also know that additional 
resources are often provided to districts that already have high achievement and 
that this can potentially mask inefficiencies in spending. 

Because of the limitations of the research, we could not evaluate the efficiency 
of a district against an external benchmark. We therefore rated districts based on 
their relative performances. That means a few things: First, we slotted districts 
into different evaluation levels even though in some cases the numerical value that 
separated the districts may not have been significant. It also means that states with 
a smaller number of districts had different cutoff points between rating categories 
than did states with larger numbers of districts. 

Our measures also cannot account for all of the variables outside the control of 
a district, in large part because the field of education suffers from a lack of high- 
quality data. Some states do not have robust school-by-school spending data, 
for instance. That is why we were able to produce only district-level productivity 
results, which likely mask significant variation within a district. And apart from 
excluding small districts, we did not adjust for economies of scale. There are 
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issues with the data^ as well as dehate within the research community about what 
economies of scale say about the quality of a district s managements^ However, 
given the potential impact that size can have on spending, we made it easy to sort 
by both enrollment and geography on our interactive website so that users can 
compare similar districts. 

The available data are also problematic. State and district data often suffer from 
weak definitions and questionable reliability. For instance, the federal government 
requires that every school report the number of students who participate in the 
free and reduced-price lunch program. But schools rely on parental self-reporting 
to determine eligibility, and so schools that are more aggressive about recruiting 
families into the program often have higher participation rates, even though they 
might not necessarily have larger percentages of low-income students. 

There are problems with achievement data as well. Many of the state assessments 
currently in place do not rigorously assess what students know and are able to do. 
Some of the exams use only multiple-choice questions to test students’ mastery 
of a subject, thus providing limited perspective on student skills. Other exams 
are not properly aligned with state curriculum standards and maybe too easy. 
Moreover, our study looks only at reading and math test scores, an admittedly 
very narrow slice of what students need to know to succeed in college and the 
workplace. 

Despite these caveats, we believe our evaluations are the best available, given exist- 
ing traditions and knowledge. We designed our color-rating system to empower 
the public to engage the issue of educational productivity, and we have produced 
an interactive website that allows users to compare the productivity of similar 
districts. We hope this project promotes not just further talk and deeper research 
but also thoughtful action to maximize school spending. 
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Findings 


This report is designed to continue to press for a national conversation about edu- 
cational productivity and to identify districts that generated more relative achieve- 
ment per dollar spent. Below are our major findings: 


Far more can be done to boost educational productivity. 

Our research suggests there is large and significant lost educational capacity in 
our school system^ and highly inefficient districts exist in almost every state, with 
more than 1 million students nationally enrolled in low-productivity districts. 
More than 275 school districts around the country were rated highly inefficient on 
all three of our productivity metrics. These districts serve about 3 percent of the 
more than 41 million students covered by our study. 

To be clear, the issue here is not that any districts are necessarily wasting money 
on their education efforts. Rather, the issue is that too many districts are spend- 
ing taxpayer money in ways that do not appear to dramatically boost reading and 
math scores, and some districts are able to gain similar levels of reading and math 
achievement with the same population of students but at lower levels of per- 
student spending. In other words, we need to do more to follow the lead of the 
top-performing districts and ensure that school dollars go to improving results. 


Some of the country's richest school systems show a clear lack of 
productivity 

Our analysis showed that after accounting for factors outside of a district s control, 
many high-spending districts posted middling productivity results. For example, 
only 37 percent of 2,397 districts in the top third in spending were also in the top 
third in achievement. (Note that the productivity rankings for 20 14 cannot be 
compared to the rankings in previous years, due to methodological limitations) 
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To cite an example, consider Rondout Valley School District in New York. In 
2011, the district spent around $25,500 per student in unadjusted expenditures, 
which puts the district in the upper tier of the state in terms of spending.'^* But the 
district gets lower achievement relative to other districts with similar demograph- 
ics, and in fourth grade math in 201 1, 58 percent of Rondout students scored 
proficient. The state average was 87 percent.'^’ This suggests a gap between what 
the districts spends and what it might actually get for those dollars. 


Too many state states and districts are spending dollars on 
programs that fail to improve student outcomes 

In the few states that do produce robust and detailed district budgets, funding 
priorities in some districts seem misguided. Texas, for instance, is one of the few 
states we found that published spending data at the state level on athletics, and we 
found that more than 100 districts in the state spent $500 or more per student on 
athletics in 2013.^° Three districts in the Lone Star state spend more than $1,000 
per student on athletics. Or consider the fact that a district could give each student 
his or her own iMac computer for that same amount of money. 

We looked more closely at these Texas districts that spend significant amounts of 
money on athletics, and many of them were highly rural districts. Given the nature 
of the data, we were not able to determine why they allocated so much on athlet- 
ics. But the results suggest that much of the costs of the athletics program went 
toward transportation costs. 

Still, some of these high-athletic spenders have middling academics. Take Crane 
Independent School District in Crane, Texas, for example. The district has a 
graduation rate of 77 percent, which is slightly lower than expected given its 
characteristics, according to one analysis.^' Moreover, the district’s test scores 
are below average in some areas, and in 20 1 1, only 48 percent of its 10th graders 
scored proficient on the state’s math test. The state average was 74 percent. And 
yet in 2013, the Crane Independent district spent more than $600 per student 
on athletics, according to the state’s fiscal data. That is not to say, however, that 
the money spent on athletics is wasted — Crane’s boy’s track team won the state 
title some years ago.^^ Rather, we would argue that some districts may have their 
priorities wrong and would be better off spending their dollars in ways that boost 
student outcomes in the classroom. 
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As Marguerite Roza^ the director of the Edunomics Lab at Georgetown 
University^ argues, districts often lack the ability or even make the effort to break- 
down per pupil spending. And Texas should be praised for divulging these data. 
Indeed, as part of our research, we looked across the states and could not find 
another state that reported this detailed level of information about athletic spend- 
ing. A handful of states such as New Jersey provided information on extracur- 
ricular spending, but even those states did not break out the data. More than that, 
most states do not provide any fiscal data on spending on extracurricular activities. 


Many states fail to track fiscal effectiveness 

Our research shows that without focused programs and policies, education spend- 
ing does not always boost test-scores. In more than half of the states included in 
our study, there was no clear relationship between spending and achievement after 
adjusting for other variables, such as cost of living and students in poverty. Still, far 
too few states track this issue and are able to show that school dollars are spent well. 

We found only two states, Florida and Texas, that regularly analyzed the produc- 
tivity of their schools and districts. What’s more, only a few states, such as Rhode 
Island, have taken a weighted student funding approach to education, which can 
boost equity and effectiveness. This is not enough, and state and districts need to 
do far more to get a better education bang for their buck. 

We examined what states have been doing to improve productivity and found 
some bright spots. A number of states, for instance, have been taking efforts 
to improve productivity since our last report. One of those states. New York, 
has taken an innovative approach to boost educational efticiency by offering 
efftciency grants to districts, as we describe above in the section on recent state 
productivity reforms. 

Virginia also continues to bolster local efforts at productivity increase by creat- 
ing a type of efftciency “SWAT” teams, as we detailed above as well. These 
teams visit local school systems and help build capacity by provide technical 
assistance. Both New York and Virginia should serve as models to other states 
looking to promote efftciency. 
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States fail to equitably fund schools and low-income students are 
more likely to be in least productive districts 


In our analysis, we calculated the expenditure difference between a district that 
has expenditures near the top — the 95th percentile — and near the bottom — the 
5th percentile — in each state. This is a long-standing approach to measuring 
school finance inequity^^ For our report, we used the most recent spending data 
available and found that the spending gaps among school districts remain high. 

In Nevada, the difference between the wealthiest districts and the least wealthy 
district was more than $6,000.^'* In Louisiana, the difference between the wealthi- 
est districts and the least wealthy district was more than $7,000.^^ 

For this reason, we took significant steps in our report to control for funding 
disparities among populations of students, yet low-productivity districts are also 
more likely to enroll students from low-income households. Like in our previous 
analysis, our study showed that students who participated in free and reduced- 
price lunch programs are twice as likely to be enrolled in the nation’s least-pro- 
ductive districts, even after making allowances for the higher cost of educating 
lower- income students. We also found similar results with students of color. 

In particular, Hispanic students are twice as likely to be in the least productive 
districts than in the most productive districts, and black students are eight times 
more likely to be in the least productive than in the most productive districts. 


State budget practices are often weak, vague, and inconsistent 

Within and across states, budgeting procedures differ widely, making it difficult 
to track expenditures. In the state of Washington, for instance, school districts are 
allowed to keep two different financial statements: One set of financial statements 
to conform with the state’s legal requirements and the other set to meet general 
accepted accounting principles.^* In an email, Daniel Lunghofer, the state’s acting 
supervisor of school district accounting, told us that state law requires that dis- 
tricts report their budgets in a way that meets state requirements. He said that the 
two sets of accounting approaches “are not a perfect match” but “they are compa- 
rable.” Whatever the case, having two sets of budgets impedes transparency. It also 
makes it difficult for outsiders to figure out if the district is spending its money 
effectively, since analysts cannot easily compare across budgets. 
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Inconsistency is a problem in Idaho as well^ and in the state^ charter districts 
and non-charter districts can use different accounting procedures'^ For people 
not deeply familiar with the accounting procedures, this makes it hard to com- 
pare spending across districts types. The budget systems are also often opaque. 
Vermont, for example, allows a district to report expenditures both with or with- 
out certain tuition and fees in the state’s annual statistical report.® 

This issue of inconsistency and transparency gets far worse once we look across 
the 50 states. While almost every state now has a common chart of accounts, the 
definitions are not comparable. This means that what might count as curriculum 
spending in one state is different than what counts as curriculum spending in 
another state. For instance, we found that some states gave clear technical defini- 
tions around certain fiscal items, while other states did not. 

Varying definitions are even an issue with the federal government’s education pro- 
grams. Take long-term debt, for example. The Department of Education’s National 
Public Education Financial Survey defines long-term debt as “revenues received 
from bond sales, other borrowing, and the sale of fixed assets.”'’^ In contrast, the 
U.S. Census Bureau, which also collects education expenditures data, defines the 
term as “debt payable more than one year after date of issue” and uses “four data 
items detailing the debt at the beginning and end of the year, as well as the debt 
issued during the year and the debt retired during the year.”® These sorts of differ- 
ences cause additional burdens for districts. It also causes confusion for educators, 
analysts, and other outside observers who examine this data. 

As we discuss in the Rhode Island example above, data alone are not powerful. 
Wbat make data powerful are how exactly they are used, and in many states, there 
is not a robust way to link fiscal data to program and outcome data. According to 
the Data Quality Campaign’s 2014 survey, 25 states cannot connect their expen- 
diture databases to the state’s longitudinal data system.® These kinds of discon- 
nects make it difficult to figure out if a program is cost-effective. More than that, 
the Data Quality Campaign reports that only two states currently can report their 
fiscal data at the classroom level.'^'^ 
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Recommendations 


Our analysis leads us to the following recommendations: 


States should build capacity for greater productivity gains through 
benchmarking, targeted grants, and assistance teams 

Currently many districts lack the capacity to do more with less. That is to say in 
a time of sagging budgets, districts lack the know-how — or even the data — to 
jumpstart productivity gains. This cannot continue, and we believe that states 
should considering providing grants that link funding increases to improved 
student achievement. As noted earlier, the state of New York has been a pioneer in 
this area. 

States should also consider following the lead of Virginia and build technical 
assistance teams that can help districts to find cost savings. States can also help 
boost capacity by providing better data. One crucial approach to improving data 
is providing districts with productivity evaluations. These sorts of performance 
metrics — when done well — can provide local leaders with more information on 
their districts, as well as guide best practice. 

However, building better data procedures alone is not enough. It is crucial that 
districts connect their data to other information such as instructional or academic 
information and build local capacity to use that information in the most effective 
and meaningful ways. The federal government can play a role in establishing new 
data systems. As Harvard Professor Martin West notes, Washington should ensure 
that data systems funded with federal money can connect programmatic data and 
fiscal data so that leaders can examine the productivity of school reform efforts.'’^ 
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Education leaders should improve accounting procedures and 
create a multistate initiative that will focus on building more 
robust education budgets 


Educators can do a lot within their communities to make accounting and budgets 
more transparent and actionable. Some states have released highly robust school- 
level fiscal databases, which make it easier to evaluate local levels of equity and 
effectiveness. Other states have built detailed state-level databases that provide 
very granular indicators on local spending. As we note above, Texas provides very 
detailed data — including information on spending on sports. 

The Common Core State Standards Initiative provides an example of how 
states can work together to create a stronger, more innovative education system. 
Something similar could be done within the fiscal space, with states coming 
together to develop more rigorous budgeting procedures. Such a group of state 
education leaders and stakeholders could create a common chart of accounts — a 
type of budget dictionary — as well as set out best practices when it comes to link- 
ing fiscal data to other databases. 

This is particularly important because fiscal data practices vary so widely across 
states, and a common set of approaches would do a lot to consolidate the field. At 
the same, this sort of process needs to make space for innovation. Over the past 
few years, education had changed dramatically. There have been major innova- 
tions around standards and assessments, for instance. New digital tools have 
also gone a long way to change how schools and districts work, and an updated 
accounting system would need to offer some flexibility for change. 


States and districts should encourage smarter, fairer approaches to 
school funding such as student-based funding policies 

Policymakers should develop funding policies that direct money to students 
based on their needs. This will go a long way to give all schools and districts an 
equal opportunity to succeed. At the same time, the gross funding inequality that 
lies between school district cannot be ignored; therefore steps must be taken to 
improve fiscal equity across schools, districts, and states. Specifically, we recom- 
mend weighted student-funding, which has the potential to solve both equity and 
efficacy issues with current school funding approaches. 
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States also need to do more to allow innovation and flexibility within funding 
programs. Today funding arrives at the district accounting office with too many 
strings attached. The reliance on categorical grants — funds to districts with strict 
limits on their use — prevents innovation. Weighted student funding can help ease 
districts’ dependency on grant money. Moreover^ states can also help by rolling- 
back policies and procedures that limit local autonomy such as class-size restric- 
tions and seat-time policies. 
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Conclusion 


The idea that social institutions should keep a close eye on improving productiv- 
ity is not new. Within the field of health care^ for instance^ observers have long 
argued that the United States needs to do more with less. And as we noted in our 
2011 report, good managers improve performance through organizational change. 
However, effective organizational change can only be achieved by using data, set- 
ting goals, and thoughtfully implementing incentive and consequence programs 
and processes to boost outcomes. 

None of this is management rocket science, to be sure. Still, in education, we sim- 
ply have not had a national conversation about what bang we get for our education 
buck. But given current trends, such as low revenues and increasing academic 
demands, we cannot continue to put off this conversation. We also recognize that 
there are myriad of other issues plaguing our school finance system — from issues 
of equity to a simple lack of good data. 

But in the end, our nation’s school finance system needs to address the productiv- 
ity issue, and the reason is simple, as we’ve argued before. Fiscal accountability 
is central to our public education system, and educators need to spend school 
dollars well, if they want more school dollars. Looking forward, then, we must 
ask ourselves: How can we do more with what we have? How can we ensure that 
each school dollar is well spent? How can we make sure all education funds work 
for students? 'The answers to these questions should be at the core of our national 
education system’s management, and they should be central to any debate over the 
future of our schools. 


Conclusion | www.americanprogress.org 35 


About the author 


Ulrich Boser is a Senior Fellow at the Center for American Progress, where he ana- 
lyzes education, criminal justice, and other social policy issues. Prior to joining the 
Center, Boser was a contributing editor for U.S. News 6’ World Report His writings 
have appeared in many publications, including The New York Times, Wall Street 
journal, and The Washington Post He is working on a book on learning. 


Acknowledgements 

The author wishes to express his gratitude for the tremendous research assistance 
of Rob Hanna. He did much of the data analysis for this report. Many others also 
did a great deal of research, including Tiffany Hsieh and Chelsea Straus. Other 
staff at the Center also offered a great deal of generous feedback and support, 
including Carmel Martin and Cynthia Brown. 


36 Center for American Progress | Return on Educational Investment: 2014 


Appendix A: 

Frequently asked questions 


In the text below, we reprinted, with slight modifications, the Frequently Asked 
Questions from the 201 1 report that apply to this report as well 

What is educational productivity? 

In the business worlds productivity is a measure of benefit received relative to 
spending. This project adopts that concept to measure public school districts’ 
academic achievement relative to their educational spending, while controlling for 
cost of living, student poverty, the percentage of students in special education, and 
the percentage of English-language learners. 

Why does CAP say that evaluations should be approached with caution? 

The connection between spending and achievement is complex, and our data 
cannot capture everything that goes into creating an efficient school system. 

Nor can we control for everything that is outside of a district’s control, and our 
adjustments for factors, such as poverty and students in special education, are 
estimations that do not account for variations in severity and type within those 
demographic groups. Additionally, some of the data reported by states and dis- 
tricts are unreliable, as agencies occasionally use inconsistent definitions and weak 
data collection practices. Therefore, while we believe our results are meaningful, 
we encourage readers to further examine the data and our approach to evaluating 
productivity, as well as caution against reading too closely into individual evalua- 
tions of districts. 

Should the United States spend less on public education? 

Our emphasis on educational productivity does not mean that we believe that 
lawmakers should spend less on education — quite the opposite. Transforming 
our schools will demand both real resources and real reform, and our project is 
an argument for dramatically improving our nation’s school system so that dollars 
create results. What’s more, we deeply believe that far more needs to be done to 
make school funding more equitable. 
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Why not create a single score for each district? 

A single score would have masked wide variation in the rankings of districts across 
our three models. We produced three productivity measures because we wanted 
to emphasize the complexity of measuring a district s efficiency and expose educa- 
tors, policymakers, and the public to different ways of measuring educational 
productivity. 

Were districts evaluated against a benchmark? 

No. We evaluated each district relative to the performance of other districts in the 
same state. That means that states with fewer districts have different evaluative 
cut points than states with larger numbers of districts. We believe this approach, 
which has been used in other education policy reports, is a fair way to evaluate 
within-state performance. 

How was achievement measured? 

We relied on data from the U.S. Department of Education, which collects achieve- 
ment data from the states on district-level student outcomes. We used these data 
to create an achievement index for each state by assigning each district a score. We 
calculated the score by adding the number of students in 20 1 1 designated profi- 
cient or above on statewide reading and math tests in third grade through eighth 
grade and in high school and divided by the number of test takers in those grades. 
The achievement index is a district s proficiency rate across these reading and 
math tests. 

How were expenditures measured? 

We used National Center for Education Statistics, or NCES, expenditure data 
from 2011, the most recent year for which complete data are available. We used 
“current expenditures,” the preferred metric among educational leaders, which 
includes salaries, services, and supplies. We did not use “total expenditures,” which 
also includes capital expenses, because these can fluctuate dramatically from year 
to year and are thus unreliable for comparisons. 

How did CAP account for differences in revenue sources? 

We did not. The fiscal database produced by NCES does not track educational 
expenditures by specific revenue source. 
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How did CAP adjust for differences in cost of living between districts? 

We used the Comparable Wage Index^ or CWI, a measure of regional variations 
in the salaries of college graduates who are not educators. Lori L. Taylor at Texas 
A&M University and William J. Fowler at George Mason University developed 
the CWI to help researchers make better comparisons across geographic areas. We 
used adjustments from fiscal year 2011, the most recent year for which data are 
available. 

Why did CAP use the percentage of students at or above the "proficient" rather than 
"basic" level to create its achievement index? 

The proficient level indicates a firm grasp of the knowledge and skills needed to 
succeed at grade level. Students scoring at the basic level have only partially mas- 
tered the necessary knowledge and skills. 

My district scores well on standardized tests, so why does it do poorly on CAP'S 
Basic and Adjusted Return on Investment Indexes? 

We rate schools on how much academic achievement they get for each dollar 
spent, while controlling for factors outside a district’s control, such as cost of liv- 
ing and students in poverty. A district therefore received high marks on our basic 
and adjusted ROl indexes if it had both high achievement and low spending rela- 
tive to other districts in the same state. Districts with high achievement and high 
spending by definition fare less well, as do districts with low achievement and low 
spending. 

My district scores poorly on standardized tests. Can It do well on CAP'S Basic and 
Adjusted Return on Investment indexes? 

No. School districts with low-student achievement cannot get a color rating higher 
than orange — or about average — on either the basic or the adjusted ROI indexes. 

My district scores poorly on standardized tests, so why does it do so well on CAR's 
Predicted Efficiency index evaluation? 

The Predicted Efficiency Index measures whether district achievement is higher 
or lower than its predicted achievement given per-pupil spending and percentage 
of students in special programs, such as subsidized school lunches. Under this 
approach, a low-achieving district could get high marks if it performed better than 
expected. 
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Can districts be compared across states? 

Because each state has its own student assessment program^ the Return on 
Investment measures listed on our wehsite are restricted to within-state compari- 
sons of districts, and comparisons of districts across states are not meaningful. 

Why is my district not included in CAR's evaluation? 

We restricted our study to districts that teach kindergarten through the 12th 
grade. We also excluded districts classified as a charter school agency state-oper- 
ated institution, regional education services agency, supervisory union, or federal 
agency. These restrictions were to ensure that districts were comparable to one 
another. We also excluded districts with inadequate demographic, achievement, or 
expenditure data or otherwise missing data. 

Why is my state not included in CAR's evaluation? 

We were unahle to produce results for Alaska, the District of Columbia, Hawaii, 
Kansas, Maine, Montana, Nebraska, New Hampshire, North Dakota, Oklahoma, 
South Dakota, and Vermont. Hawaii and Washington, D.C. are single-district 
jurisdictions, so within-state comparisons were not possible. Alaska, Arizona, 
Kansas, Maine, Montana, New Hampshire, North Dakota, Oklahoma, South 
Dakota, and Vermont did not have enough comparable districts. 
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